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Early bilingual exposure, especially exposure to two languages in different modalities such 
as speech and sign, can profoundly affect an individual's language, culture, and cognition. 
Here we explore the hypothesis that bimodal dual language exposure can also affect the 
brain's organization for language. These changes occur across brain regions universally 
important for language and parietal regions especially critical for sign language (Newman 
et al., 2002). We investigated three groups of participants (N = 29) that completed a word 
repetition task in American Sign Language (ASL) during fNIRS brain imaging. Those groups 
were (1 ) hearing ASL-English bimodal bilinguals (n = 5), (2) deaf ASL signers {n = 7), and 
(3) English monolinguals naive to sign language (n = 17). The key finding of the present 
study is that bimodal bilinguals showed reduced activation in left parietal regions relative 
to deaf ASL signers when asked to use only ASL. In contrast, this group of bimodal signers 
showed greater activation in left temporo-parietal regions relative to English monolinguals 
when asked to switch between their two languages (Kovelman et al., 2009). Converging 
evidence now suggest that bimodal bilingual experience changes the brain bases of 
language, including the left temporo-parietal regions known to be critical for sign language 
processing (Emmorey et al., 2007). The results provide insight into the resilience and 
constraints of neural plasticity for language and bilingualism. 



Keywords: language, bilingualism, brain plasticity, functional near infrared spectroscopy brain imaging, fNIRS, 
"neural signature hypothesis" 



INTRODUCTION 

Studying bilinguals for whom sign language is one of their 
two languages provides a better way of understanding both the 
resilience and plasticity of human language and the underlying 
brain regions that support it. For example, left hemisphere 
inferior frontal and temporal regions demonstrate resilience in 
their support of language functions despite variations in language 
modality (signed or spoken) or the number of languages used 
(monolingual or bilingual; Petitto et al, 2000; MacSweeney et al., 
2002a,b; Corina et al., 2003; Penhune et al, 2003; Kovelman 
et al, 2009; Emmorey et al., 2011). In contrast, the parietal brain 
regions, classically associated with visuo-spatial processing, show 
evidence of plasticity and become specialized for processing 
sign language structure (Neville et al., 1997; Newman et al., 
2002; Emmorey et al., 2005). However, our understanding of 
how the brain accommodates sign language and bilingualism is 
incomplete without a better appreciation of bimodal bilinguals, 
bilinguals who can both hear a spoken language and learn a 
sign language, from early life (Petitto et al., 2000; Emmorey and 
McCuUough, 2009). Here we explore the resilience and plasticity 
of the brain's organization for human language as an outcome 



of early bilingual language experience to a sign and a spoken 
language. 

SIGN LANGUAGES 

Sign languages are visuo-spatial languages. Researchers have been 
investigating how early exposure to a sign vs. a spoken lan- 
guage may impact the brain. Bilateral parietal regions, known to 
be important for analyzing body motion and visuo-spatial ori- 
entation in hearing monolinguals, appear to become selectively 
engaged for sign language morpho-syntax (Corina et al., 1999), 
lexical retrieval (Emmorey et al., 2011), and word production 
(Emmorey et al., 2007). Importantly, there is also evidence of 
neuro-developmental plasticity, only adults who received early 
exposure to a sign language show right parietal recruitment dur- 
ing signing (Neville et al., 1997; Newman et al, 2002). Hence, 
researchers have suggested that learning a sign language across 
both deaf and hearing individuals may change the functional- 
ity of the parietal regions supporting sign language processing 
(Neville et al, 1997; Corina et al, 1999; Newman et al., 2002). 
Yet, it also remains possible that there are differences in functional 
language organization between deaf and bimodal signers in these 
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resilient regions for language. These resUient areas include areas 
around classic Broca's and Wernicke's regions, as well as the 
plastic/parietal regions that support sign language use. 

BILINGUALISM 

Neuroimaging research comparing bilingual and monolingual 
speakers of different languages generally support the idea that 
a "bilingual is not two monolinguals in one person," (Grosjean, 
1989; e.g., Abutalebi et al, 2001; Kovelman et al, 2009; Bialystok 
et al, 2012; Jasinska and Petitto, 2013; Klein et al, 2014). A key 
notable difference between bilinguals and monolinguals is the 
increase in greater gray matter density in left parietal regions in 
unimodal spoken language bilinguals relative to monolinguals 
(Mechelli et al., 2004). That study found that high proficiency 
bilinguals who learned their two languages before the age of 5 
had the greatest increase in gray matter density relative to low 
proficiency and later exposed bilinguals, as well as monolinguals 
(Mechelli et al., 2004). Several studies have shown that unimodal 
spoken language bilinguals also typically show greater signal 
intensity in left inferior frontal regions during lexico-semantic 
(Kovelman et al., 2008c), syntactic (Jasinska and Petitto, 2013), 
and language switching tasks (Abutalebi et al., 2008; Kovelman 
et al, 2008c), relative to monolinguals. Yet, it remains generally 
unknown if these differences predominantly stem from frequent 
competition between bilinguals' two languages (Abutalebi et al, 
2008) or the overall increase in language capacity to accommo- 
date for two linguistic systems (Kovelman et al, 2008a). Research 
into bimodal bilinguals who potentially experience reduced com- 
petition between their two languages in non-overlapping auditory 
and visual modalities may shed new light on the nature of the 
brain's accommodation for two languages. 

BIMODAL BILINGUALISM 

There are many critical similarities between bimodal and uni- 
modal spoken language biHnguals. Specifically, young learners of 
a sign and a spoken language reach their first word and other lin- 
guistic milestones at the same time as unimodal bilinguals (Petitto 
et al., 2001a; Petitto and Kovelman, 2003; Petitto, 2005). Young 
bimodal and unimodal bilinguals also show similar trajectories 
of vocabulary growth across their two languages in the first 3 
years of life (Holowka et al., 2002). Regarding language switch- 
ing, bimodal bilinguals can use both languages at the same time 
while unimodal bilinguals can only use their two languages in 
alternation (because both languages cannot be expressed simul- 
taneously from the mouth). On the other hand, it has been 
found that young bimodal bilinguals show unimodal-like pat- 
terns of language switching in response to their family's switch- 
ing habits, interlocutor's language, and the morpho-syntactic 
structures of their languages (Petitto et al., 2001a; Petitto and 
Kovelman, 2003; Paradis and Nicoladis, 2007). Hence, one can 
hypothesize that all types of bilingualism, bimodal and uni- 
modal, have a similar impact on language organization in the 
brain. 

Nevertheless, it is possible that bimodal bilingualism makes a 
lesser impact on the bilingual brain relative to unimodal bilin- 
gualism. For instance, theories of bilingual cognition have put 
forth a hypothesis and supporting evidence that the necessity 



to switch between two languages from early in life can enhance 
a bilingual child's and adult's attentional capabilities (Bialystok, 
2001, 2008; Bialystok et al, 2012; Garbin et al, 2010). However, 
this effect has not been found in young bimodal bilinguals 
(Emmorey et al., 2008), possibly because a sign and a spoken lan- 
guage compete less with each other over the sensorimotor mech- 
anisms of language production and comprehension (Bialystok, 
2001; Emmorey et al., 2008). 

Another typical point of competition between two languages 
is lexico-semantic encoding of homophones, or words that sound 
the same across languages but can often mean different things. 
Specifically, plain and plane are English homophones that sound 
the same but have different meanings, the same phenomenon 
is possible across bilinguals' two different languages (Finkbeiner 
et al., 2006; Blumenfeld and Marian, 2013). Such type of language 
competition is not possible for bimodal bilinguals. Given the evi- 
dence that there is reduced levels of competition between sign 
and spoken languages relative to two spoken languages, could it 
then be possible that bimodal bilingualism does not change the 
brain beyond the specific accommodations for the person's sign 
and spoken languages? 

What impact might dual language experience and language 
switching have for a bilingual brain? Neuroimaging studies of 
bilingual language switching have found greater activation in 
bilateral prefrontal and left temporo-parietal regions during lan- 
guage switching tasks in unimodal bilinguals as compared to 
using only one language at a time (Hernandez et al., 2001; 
Abutalebi et al, 2008; Kovelman et al, 2008c; Garbin et al, 
2010). The only study that has investigated language switch- 
ing in bimodal bilinguals found that this group only showed 
greater activation in left temporo-parietal regions during the 
switching condition as compared to non-switching and as com- 
pared to English monolinguals (Kovelman et al., 2009). In that 
study, we hypothesized that increased activation in temporo- 
parietal regions in bimodal bilinguals stemmed from the competi- 
tion between bimodal bilinguals' lexico-semantic representations 
in the two languages (Kovelman et al, 2009). Nevertheless, it 
remains possible that increased activation in left the temporo- 
parietal region was due to the engagement of sign-specific lan- 
guage processes in the parietal regions (Newman et al., 2002), 
rather than the switching demands. The limitation of the previ- 
ous study was that it only compared bimodal bilinguals to spoken 
language monolinguals. Hence, to fully answer the question on 
how the brain accommodates to bimodal bilingualism it is critical 
to also compare bimodal bilinguals to Deaf signers. 

INNOVATION OF THE PRESENT WORK 

Most of the previous research examined the impact of early sign 
language exposure as compared to late sign language exposure 
or spoken language exposure (e.g., Neville et al, 1997; Newman 
et al., 2002). Here we explore the impact of early bimodal bilingual 
exposure to a sign and to a spoken language in hearing adults as 
compared to early ASL sign language exposure in deaf adults. The 
goal of the present study is to broaden our understanding of the 
resilience and plasticity of the brain's organization for language in 
light of both bilingual and bimodal exposure to two languages in 
different modalities. 
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HYPOTHESIS 1 

There is limited sensory-motor and lexical competition between 
sign and spoken languages because they do not compete for 
the same sensory-motor apparatus and do not use similar pho- 
netic features to encode words. Thus, the bimodal bilingual brain 
requires little additional neural adjustments for language process- 
ing beyond the plasticity necessary for accommodating the given 
spoken and sign languages. Therefore, one can predict similari- 
ties across ASL-English bilingual signers and unimodal deaf ASL 
signers. 

HYPOTHESIS 2 

Early bilingual exposure will lead to evidence of the brain's neural 
accommodation for processing two language systems, both in the 
brain's resilient regions that are universal for language process- 
ing (across sign and speech, including left frontal and temporal 
regions) and the plastic regions specific to sign language process- 
ing (including the parietal regions). Here, the neural conditions 
that give rise to being bilingual (the neural demands/impact of 
processing two languages) are governing the extent to which 
bUinguals will recruit brain regions that demonstrate universal 
resilience and language-specific plasticity. Thus, we can predict 
differences across bilingual ASL-English signers and functionally 
monolingual deaf ASL signers during the unimodal use of sign 
language. 

The present study addressed these two hypotheses by testing 
three groups of participants: (1) native hearing ASL-English pro- 
ficient bimodal bUinguals from birth, (2) native deaf signers of 
American Sign Language (ASL) who learned English as a second 
language but who are functionally monolingual deaf ASL signers, 
and (3) hearing English monolinguals naive to sign. The partic- 
ipants completed a lexical/word repetition task in sign language 
which engages both language comprehension and language pro- 
duction mechanisms. This task was chosen because it is known 
to engage frontal, sensory-motor, temporal, and parietal brain 
regions known to demonstrate both resilience and plasticity in 
their functional organization for bilingualism and sign language 
exposure (Mechelli et al., 2004; Abutalebi et al, 2008; Kovelman 
et al, 2009; Emmorey et al, 2011). Participants' brain activity 
was recorded with functional Near Infrared Spectroscopy (fNIRS) 
while they completed the task. fNIRS is well suited for studies 
of higher cognition because it has both good temporal sampling 
(10-50 Hz) and good spatial resolution (~3 cm depth) without 
being in an enclosed structure (Huppert et al, 2009; Shalinsky 
et al, 2009). Hence, the two critical innovations of the study is 
the comparison between deaf and hearing bimodal bilinguals and 
the use of fNIRS technology to study sign language production in 
its ecologically natural form, that is, when participants are upright 
and using their arms and hands freely. 

MATERIALS AND METHODS 
PARTICIPANTS 

Three (3) participant groups included 17 hearing, native English 
monolinguals with no knowledge of signed language, 7 deaf 
native ASL signers, and 5 hearing native ASL-English bimodal 
bilinguals from birth (see Assessments below and Table 1). Five 
of the 7 deaf native ASL signers were congenitally deaf while two 



lost hearing before age 1. Five of the seven deaf native ASL signers 
first learned ASL from deaf family members, including parents 
and siblings. For the remaining two participants from this group, 
the primary source of ASL was preschool for the deaf between 
the ages of 1-3. The 7 deaf native ASL signers had been exposed 
to English as a second language at school and had overwhelm- 
ingly rich ASL daily language exposure, use, and maintenance 
throughout adulthood. AH 7 deaf native ASL signers considered 
themselves to be functionally dominant ASL users. All partici- 
pants were right handed, using Edinburgh Handedness Inventory 
(Oldfield, 1971). The participants in this study were also the 
participants examined in a separate study (Kovelman et al., 2009). 

Participant assessments 

All participants completed a previously standardized and pub- 
lished Language Background and Use Questionnaire that allowed 
us to assess their monolingual and/or bilingual language acquisi- 
tion history and language use. All bilingual participants reported 
early (before age 5) and systematic exposure to both languages. 
All participants also reported continual use of the two languages. 
Participants also completed a previously published Language 
Competence/Expressive Proficiency (LCEP) test (for details on 
test administration and scoring see Kovelman et al., 2008a,b,c; 
Berens et al., 2013). LCEP builds upon measures of sign language 
competence, originally established for the study of Nicaragual 
Sign Language (Senghas and Kegl, 1994), and was expanded for 
use with other sign and spoken languages (Petitto et al., 2001a; 
Schembri and Johnston, 2004; Jasinska and Petitto, 2013). All par- 
ticipants were required to score >80% on the LCEP in each of 
their native languages to be eligible for this study — thus demon- 
strating comparably high language competence in each of their 
languages. The study was approved by the Internal Ethical Review 
Board of Dartmouth College (Hanover, New Hampshire, USA). 
All participants were students and professionals in a local college 
township area. 

BRAIN IMAGING TASK 

An ASL sign repetition task was used as our imaging task. This 
is a standard task in sign modeled after the classic word repe- 
tition task. The task allows for the study of sign production in 
both native signers and individuals naive to sign languages (cf. 
Petitto et al., 2000). The participants watched a video recording 
of a native ASL signer producing ASL Real signs and Pseudosigns, 
one at a time. Pseudosigns obey the linguistic rules for phono- 
logical and lexical formation in ASL but do not exist in ASL, 
and are analogous to pseudowords in English. After each sign 
or pseudosign presentation, the participants repeated the sign 
or pseudosign. This was a block design with stimuli blocked by 
condition [sign/pseudosign, 12 blocks per condition, 5 trials per 
block (30 s blocks), 6 s per trial]. 

Real signs were meaningful, single-handed, high-frequency 
nouns. These were based on the corpus of signs commonly 
acquired by young ASL speakers before the age of 6 (Petitto 
et al, 2001a; Holowka et al., 2002; Petitto and Kovelman, 2003). 
Pseudosigns consisted of hand movements that were meaningless, 
sign-phonetic units that were syllabically organized into possible 
but non-existing short syllable strings. Like spoken languages, all 
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words in signed languages are formed from a finite set of mean- 
ingless units, called phonetic units (for example, an unmarked, 
frequent phonetic unit in sign involves a clenched fist with an 
extended thumb) . To ensure ecological validity of the pseudoword 
task, we used real but meaningless phonetic units, including 
those documented in ASL-exposed infants during manual bab- 
bling (Petitto and Marentette, 1991; Petitto et al, 2001b). These 
phonetic units were further organized into syllables (e.g., specific 
hand shapes organized into temporally constrained, rhythmic- 
movement-nonmovement alternations). Rigorous psycholinguis- 
tic experimental procedures were used to pilot the stimuli to 
ensure that they indeed contained true meaningless phonetic and 
syllabic units. For more details about these methods and the stim- 
uli please see Petitto et al. (2000); for video demonstration of the 
stimuli please see supplementary PNAS methods or visit http:// 
petitto . gallaudet.edu/ 



holding the fNIRS probes while positioned comfortably in a 
reclining chair, with the fiber optics hanging loosely without 
making contact with the body or chair (Figure lA). 

fNIRS data was recorded at 10 Hz and high-pass filtered at 
0.5 Hz to remove physiological noise (especially heart rate). The 
modified Beer-Lambert equation (mBL; Delpy et al., 1988) was 
used to convert wavelength data to oxy- and deoxy-hemoglobin 
concentrations. The application of the mBL was conducted in 
two steps. Under the assumption of constant scattering over the 
path length, the attenuation for each wavelength (A(t)) was cal- 
culated by comparing the optical density of light intensity during 
the task (Itask) to the calculated baseline of the signal (Ibaselhw)- We 
then used these A values for each wavelength and sampled time 
point (f) to solve the modified Beer- Lambert equation (Equation 
1; for more information see Huppert et al., 2009; Shalinsky et al., 
2009). 



DATA ACQUISITION, ANATOMICAL COREGISTRATION, AND fNIRS 
SIGNAL ANALYSIS 

The accurate neuroanatomical placement of fNIRS probes and 
the confirmation of ROIs (regions of interest) were achieved by 
using the 10-20 system (Jurcak et al., 2007; Shalinsky et al., 
2009). The 10-20 anchor points were established using pub- 
lished neuroanatomical landmarks for the 10-20 system (Koessler 
et al, 2009) as well as by obtaining MRI scans for a subset of 
participants {n = 9; Figures 1B,C). Stereotactic localization of 
the probe array was further confirmed for each participant by 
using a Fasttrak spatial detection system (Polhemus, Colchester, 
VT). During fNIRS scanning, participants wore a soft headband 



AA;,(f) = log 



10 
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deoxy oxy 
deoxy oxy 



oxy^,^ 



(1) 

(2) 



'^i^deoxy, '^i^deoxy, '^i^deoxy, and >^2Eoxy are the Constants for 
the extinction coefficients that measure the fraction of light 
lost to absorption per unit concentration distance in the tissue 
(Equation 2). The resultant Cdeoxy and Coxy values are the concen- 
trations of deoxygenated and oxygenated hemoglobin for each f. 



Table 1 | Participant groups' ages and language background and participant information. 



Group 



Mean age (range) 



Parents' native 
language(s) 



Age of language 
exposure 



Eng 



ASL 



Mean performance on language 
proficiency tasks 



Eng 



ASL 



English monolinguals n = 17 (10 female) 19 (18-25) English only 

Deaf ASL signers n = 7 (4 female) 26(19^2) ASL English both 

ASLEnglish bilinguals n = 5 (4 female) 24 (16-32) ASL English both 



Birth 



Birth 



Birth-4 years 
Birth 



96% 



96% 



100% 
98% 




FIGURE 1 I Hitachi ETG-4000 Imaging System MRI Neuroanatomical 
Co-Registration. (A) Participant with a Hitachi 48-channel ETG-4000. 
Optodes in place and ready for data acquisition. The 3x5 optode arrays were 
positioned on participants' heads using rigorous anatomical localization 
measures including 10 x 20 system and MRI coregistration. (B) MRI 



co-registration was conducted by having nine healthy adults wear two 3x5 
arrays with vitamin-E capsules in place of the optodes in MRI. (C) Anatomical 
MRI images were used to identify the location of optodes (Vitamin E 
capsules) with respect to underlying brain structures (sagittal section shown 
here). Location of the fNIRS channels numbered in yellow. 
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Cdeoxy and Coxy values for each channel were then plotted and 
inspected. The maximum positive or negative peak values (for 
oxy and deoxy, respectively) were determined for each channel 
from 5 s after the onset of the trial until the end of the trial. To 
compute the baseline values of oxy- and deoxy-hemoglobin con- 
centrations, we took the mean of the 5 s-segment leading up to 
the start of the trial (for more detail on pre and post-processing 
analytical options for fNIRS, see Huppert et al, 2009; Shalinsky 
et al, 2009; Gervain et al, 2011; Tak and Ye, 2014). These values 
were then used in the statistical analysis. 

First, we conducted a principle component analysis (PCA) 
to explore any differences in the pattern of brain activity 
between signers (Deaf and Bimodal) and non-signers (monolin- 
gual English speakers). Potential differences in the left hemisphere 
typically associated with language processes in speech and in sign 
were of critical interest (Petitto et al, 2000), and hence the PCA 
analyses were limited to the left hemisphere's channels (Figure 2). 
Second, we used anatomical information from MRI probe co- 
registration (Figure 1), in combination with present and prior 
PCA findings for sign and spoken languages (Kovelman et al., 
2009), to reduce the 22 channels per hemisphere to 7 regions of 
interest for each hemisphere. 

Finally, to examine differences between bimodal and Deaf 
signers we conducted repeated measures ANOVAs with follow-up 
Mann-Whitney f-test for non-related samples. The analyses were 
conducted for each hemisphere separately, while using two types 
of comparisons: one with Real and Pseudosign treated as separate 
factors, and one with both conditions averaged to optimize the 
study's power given the low number of participants. 

RESULTS 

BEHAVIORAL RESULTS 

Native ASL signers scored participants' accuracy on the basis 
of the phonological constituents of sign, hand-shape and hand- 
movement, for each sign and pseudosign (Petitto et al., 2000). 
Both groups received training with the task, as it was impor- 
tant that the groups be as accurate as possible with their hand 
movements, while maintaining good head-posture. As a result, 
all groups showed a relatively high task performance (Table 2). 



PCA Results 



■ Deaf & Bimodal ASL Signers 
c English Monolinguals 



10 12 13 1 



15 15 17 18 19 



Left Hemishere Channels 



FIGURE 2 I PCA results for the first component for the left hemisphere 
channels during the Real Sign condition. The first component accounted 
for 41 % variance in English monolinguals and 30% variance in Deaf and 
Bimodal ASL signers. 



We used a 3 x 2 mbced-measures ANOVA to compare language 
groups (English monolinguals, bimodal bilinguals, deaf ASL 
signers; between group factor) across two language conditions 
(Signs vs. Pseudosigns; within group factor) and found signifi- 
cant effects of group [f(2, 24) = 6.7, p = 0.005], task 24) = 
8.9, p = 0.006] and the interaction [f(2, 24) = 3.5, p = 0.04]. 
Post-hoc comparisons revealed a significantly better performance 
in Deaf Signers relative to English monolinguals (Tukey's HSD, 
p < 0.05), but no significant differences between the two hear- 
ing groups (p > 0.05). Additional follow-up paired f-test com- 
parisons revealed that Deaf signers [f(6) = 3.6, p = 0.01] and 
Bimodal Bilinguals [t(5) = 2.8, p = 0.07], but not English mono- 
linguals ]f(i6) = — 1.9,p = 0.85], were more accurate in repeating 
the Real signs than Pseudosigns (Table 2). 

fNIRS RESULTS 
PCA analysis 

In native ASL signers, PCA yielded a first component that 
explained 41% of the variance in English monolinguals and 30% 
variance in native signers (across Deaf and Bimodal signers). 
The pattern of the positive/negative loading for the first com- 
ponent was nearly the opposite between signers and non-signers 
(Figure 2). The negative loadings in signers (and positive in non- 
signers) appeared to span superior frontal and sensory-motor 
regions. The positive loadings for signers (and negative for non- 
signers) appeared to span inferior frontal and occipito-temporal 
regions. 

The contrasting pattern of PCA loadings suggested that the 
two groups treated the task differently, possibly due to the lin- 
guistic vs. non-linguistic nature of sign processing in Deaf and 
bimodal signers, relative to English monolinguals (Petitto et al., 
2000; Emmorey et al., 2005). Hence, all of our subsequent analy- 
ses only included native signers because our primary target was 
language processing in bilinguals and monolinguals. Moreover, 
the differences or similarities in mental processes between signers 
and non-signers might not necessarily be reflected by differences 
in the strength of activation in regions such as IFG, which are 
thought to support both language (Poeppel et al, 2012) and 
movement imitation (Kilner et al., 2009). 

Brain regions of interest 

The study aimed to bridge findings between prior bilingual 
research on spoken languages with present findings for sign 
language. Hence, to ensure that the results were maximally com- 
parable across past and present findings, the channel grouping 
into specific regions was governed by previously published group- 
ing for this same set of participants (same group of participants 



Table 2 | Participants' task performance. Mean (Standard Deviation) 
behavioral scores measured in percent correct for the sign repetition 
task by language group. 



Group 

Deaf ASL signers 
Bimodal bilinguals 
English monolinguals 



Real signs 

99 (0.01) 
98 (0.03) 
89 (0.06) 



Pseudosigns 

93 (0.04) 
90 (0.09) 
89 (0.07) 
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but a comparison of English language task between bimodal 
bilinguals to English monolinguals; Kovelman et al, 2009), MRI 
probe coregistration (Figure 1) and the PCA results. The chan- 
nels were grouped into seven regions as follows, (i) Inferior 
frontal/anterior STG (BA 44/45 and 38/22, channels 15, 19, 20, 
overlapping with the classic Broca's area). Please note that we rec- 
ognize that IPG and anterior STG are two distinct anatomical 
areas. However, since channels overlaying these areas consistently 
clustered together in every PCA analysis (including sign repe- 
tition, see below), we combine these areas into one data- and 
prediction-driven functional/anatomical ROI. (ii) Dorsolateral 
prefrontal cortex (DLPFC) (BA 9/46, channels 10 and 14), (iii) 
superior frontal (BA 9/10, channels 1,5), (iv) posterior temporal 
(BA 21/22, channel 17, overlapping with classic Wernicke's area), 
(v) parietal (including inferior and superior parietal regions, BA 
7/39/40; channels 4, 8, 9, 13), (vi) sensory-motor (BA 1-6, chan- 
nel 7), and (vii) premotor (BA 9/6; channels 6 and 11). Average 
and standard deviations of signal changes for each ROI, each 
group and each experimental condition are presented in Table 3. 

Brain bases of sign repetition in bimodal bilinguals and deaf 
signers 

First, we explored potential interaction between participants' 
brain activation and the type of experimental condition (sign 
vs. pseudosign) using a 2 x 2x 7 mixed measures ANOVA 
(groups X task x all brain regions), conducted separately for 
each hemisphere. Left hemisphere analyses revealed a marginally- 
significant task X group interaction [-F(i, eo) = 3.7, p = 0.08], 
and a marginally-significant main effect of group [_F(i lo) = 4.2, 
p = 0.06]. There were no significant main effects of task {p = 
0.36) or brain regions (p = 0.13). To identify the source of group 
X task interaction, we conducted two 2x7 mixed measures 
ANOVAs (groups x regions), one for Real and one for Pseudosign 
conditions. Group differences remained significant only during 



the Pseudosign condition [_F(i^ lo) = 9.7, p = 0.01], with Deaf 
signers showing overall greater activation during Pseudosign con- 
dition, especially in left MFG [f(io) = -2.2, p = 0.03] and SFG 
[f(iQ) = — 1.9,p = 0.07] regions. In sum, while the two groups of 
participants showed overall similar levels of activation during the 
Real Sign condition, this activation increased in Deaf signers dur- 
ing the Pseudosign condition, but decreased or stayed the same in 
bimodal bilinguals (Figure 3). Right hemisphere analyses did not 
yield any significant effects. 

Second, we explored the possible presence of overall activation 
differences between bimodal and Deaf signers, differences that 
may span both Real and Pseudosign conditions. For this compar- 
ison we conducted two 2x7 mixed measures ANOVAs (groups 
X regions), with activation averaged across Real and Pseudosign 
conditions, separately for each hemisphere. Left hemisphere anal- 
yses revealed marginally significant group differences [-F(i_ lo) = 
4.2, p =0.06] and no main effects of region (p = 0.13) or interac- 
tions (p = 0.66). Bimodal bilinguals showed lower overall levels 
of left hemisphere activation relative to Deaf signers, the differ- 
ences reached significance in the left parietal region [f(io) = —2.0, 
p = 0.03; Figure 3]. Right hemisphere analyses did not yield any 
significant effects. The results are also summarized in Figure 3. 

DISCUSSION 

The study investigated the extent of resiliency ("fixed" or sta- 
ble brain structures, systems, and their functions) and plasticity 
(neural plasticity of brain structures, systems, and their func- 
tions) of human language organization in the brain. We used 
fNIRS imaging to study native hearing ASL-English bimodal 
bilinguals as compared to native Deaf ASL signers to investigate 
the nature of bilingual brain organization for language. 

Relative to spoken language monolinguals, unimodal 
bilinguals have been shown to demonstrate different levels 
of brain activation across left frontal, sensory-motor, and 



Table 3 | Participants' brain activation. Mean (Standard Deviation) brain activation as measured in fnirs percent signal intensity during sign 
repetition task by condition and brain region. 

Group Condition IFG/anterior STG Posterior DLPFC Superior Parietal Posterior Sensory-motor 

temporal frontal frontal/motor 

BA 44/45 and 21 BA 21/22 BA 9/46 BA 1-5 

LEFT HEMISPHERE '^^^^^^■■■■■■i^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^M 



English monolinguals 


Real sign 


0.42 (0.27) 


0.28 (0.17) 


0.38 (0.16) 


0.31 (0.16) 


0.42 (0.18) 


0.35 (0.19) 


0.42 (0.26) 




Non-sign 


0.50 (0.21) 


0.32 (0.17) 


0.46 (0.22) 


0.33 (0.23) 


0.32 (0.20) 


0.40 (0.20) 


0.47 (0.27) 


Deaf ASL signers 


Real sign 


0.42 (0.24) 


0.22 (0.07) 


0.37 (0.16) 


0.37 (0.27) 


0.35 (0.13) 


0.39 (0.25) 


0.37 (0.31) 




Non-sign 


0.39 (0.19) 


0.43 (0.29) 


0.39 (0.15) 


0.43 (0.18) 


0.44 (0.17) 


0.45 (0.16) 


0.52 (0.25) 


ASLEnglish bilinguals 


Real sign 


0.52 (0.28) 


0.20 (0.06) 


0.31 (0.19) 


0.28 (0.08) 


0.22 (0.09) 


0.37 (0.13) 


0.41 (0.12) 




Non-sign 


0.46 (0.17) 


0.24 (0.17) 


0.23(0.14) 


0.25 (0.12) 


0.29 (0.12) 


0.30 (0.20) 


0.36 (0.22) 




English monolinguals 


Real sign 


0.36 (0.16) 


0.26 (0.19) 


0.38 (0.24) 


0.35 (0.19) 


0.42 (0.22) 


0.36 (0.22) 


0.43 (0.19) 




Non-sign 


0.37 (0.25) 


0.26 (0.23) 


0.33 (0.18) 


0.35 (0.26) 


0.35 (0.18) 


0.42 (0.21) 


0.41 (0.27) 


Deaf ASL signers 


Real sign 


0.39 (0.20) 


0.34 (0.21) 


0.43 (0.17) 


0.46 (0.26) 


0.40 (0.15) 


0.29 (0.19) 


0.32 (0.22) 




Non-sign 


0.44 (0.27) 


0.30 (0.18) 


0.42 (0.25) 


0.43 (0.25) 


0.41 (0.14) 


0.26 (0.13) 


0.36 (0.21) 


ASL-English bilinguals 


Real sign 


0.51 (0.19) 


0.28 (0.14) 


0.39 (0.27) 


0.40 (0.27) 


0.31 (0.16) 


0.39 (0.16) 


0.55 (0.27) 




Non-sign 


0.45 (0.19) 


0.39 (0.21) 


0.28 (0.15) 


0.36 (0.14) 


0.41 (0.19) 


0.30 (0.15) 


0.41 (0.22) 
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FIGURE 3 I Brain activation in Deaf and Bimodal ASL users during Sign and Pseudosign conditions for all left hemisphere brain regions. Error bars 
represent standard error of the mean. 



temporo-parietal regions of the brain (Kovelman et al., 2009; 
Jasinska and Petitto, 2013). Bilingual theories suggest that 
the need to resolve competing sensory-motor and lexical 
representations drives many of these neurodevelopmental 
changes (Abutalebi et al, 2008; Garbin et al, 2010). It is 
possible that such effects are reduced in bimodal bUinguals 
because their two languages are less likely to compete for 
sensory-motor production and encoding (Emmorey et al., 
2008). It has been previously found that bimodal bUinguals 



show greater activation in left temporo-parietal regions during 
language switching relative to their unimodal use of English 
or to English monolinguals (Kovelman et al., 2009), but it 
remained unclear if this was due to bilingualism or exposure to 
sign languages, which places special demands on these regions 
(Newman et al., 2002). The present findings revealed multiple 
ways in which the left hemisphere brain activation in bimodal 
bilinguals differed from Deaf signers, including differences 
in the left parietal regions. These data suggest that there is 
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indeed an interaction between bilingual and sign language 
exposure on the bimodal bUingual's language organization in the 
brain. 

The behavioral data revealed that during the Sign Repetition 
task, the two sign-exposed groups had comparable language com- 
petence and experimental task performance. In contrast, the neu- 
roimaging data revealed information that behavioral data alone 
could not. First, we found that bilingual exposure to sign and 
speech had a broad spectrum effect on the bilingual's brain bases 
of language processing throughout the left but not the right hemi- 
sphere regions. Specifically, pseudowords are thought to incur 
greater effort on parts of the language mechanisms (and their 
neural correlates) that participate in word search and retrieval 
(Zatorre et al, 1996; Petitto et al, 2000; Xiao et al, 2005; Koenigs 
et al., 2011; Perrachione et al., under review). Yet, the findings 
suggest that Deaf signers showed a stronger and more widespread 
increase in brain activation during pseudo-sign relative to Real 
sign conditions (Figure 3). As a result, there was a significant dif- 
ference in overall activation between the two groups during the 
pseudo-sign condition. 

The interaction effect for the Real sign vs. Pseudosign con- 
dition was especially pronounced in the MFG and SFG frontal 
regions of the left hemisphere. Specifically, Deaf signers showed 
significantly greater activation in the two regions relative to bilin- 
gual bimodal signers. These left middle and superior frontal 
regions are typically associated with verbal working memory 
and other aspects of executive functioning that support overall 
language processing (Adleman et al., 2002). Repetition of a pseu- 
doword requires one to perceive, encode, maintain, and repeat an 
unfamiliar linguistic unit that cannot be retrieved from memory 
(Petitto et al., 2000). Hence, it is possible that Deaf signers showed 
greater activation in these regions because of the increased work- 
ing memory load for unfamiliar words. For bilingual participants, 
these regions also form the frontal lobe aspect of the bilingual 
switching mechanism (Abutalebi et al, 2008; Kovelman et al., 
2008c). One possibility is that dual language switching experi- 
ences have enhanced the functionality of these frontal regions 
in bilinguals, such that both familiar and unfamiliar words exert 
similar amounts of effort on the part of these frontal regions. 

Another possibility is that the bilingual brain is better adapted 
to encountering and learning new words. Prior bilingual stud- 
ies have shown that bilinguals tend to have lower vocabularies 
in each of their languages relative to monolinguals, likely due 
to varied types of language experiences across the bilinguals' 
two languages (Petitto et al, 2001a). Hence, encountering and 
encoding new words might be a more common-place experi- 
ence for bilinguals, resulting in lower effort on the part of the 
frontal lobe regions for this task. Overall, the increase of bilat- 
eral frontal lobe activation is typically found during bilinguals' 
non-native language processing (Kim et al, 1997; Wartenburger 
et al, 2003) or language switching (e.g., Hernandez et al., 
2001; Abutalebi et al., 2008). Yet, less is known about bilin- 
gual brain functioning during the regular unilingual use of their 
first language. Our data suggests that such bilingual brain acti- 
vation might be reduced relative to monolinguals, especially 
during more demanding language tasks, such as processing of 
pseudowords. 



Prior research comparing bimodal bilinguals to English mono- 
linguals has shown greater activation in the left temporo-parietal 
regions in bimodal bilinguals (Kovelman et al., 2009). Yet, it 
remained unclear if this was due to the bilingual exposure in 
general or sign language exposure in particular, given the crit- 
ical importance of the parietal regions for native-like linguistic 
processing of sign languages (Neville et al., 1997; Corina et al, 
1999; Petitto et al., 2000; Corina and McBurney, 2001; Newman 
et al, 2002; Emmorey et al., 2007, 2011). The present findings 
suggest that hearing native ASL-English bimodal bilinguals had 
reduced neural recruitment of the left parietal region relative to 
the deaf native ASL users. The findings are thus parallel to spo- 
ken language findings in bimodal bilinguals: the two languages 
of bimodal bilinguals (sign and speech) show less activation in 
left temporo-parietal regions during the unimodal use of only 
one of the two languages relative to their use of two languages 
simultaneously or in rapid alternation. Hence, we suggest that 
this finding is not explained by sensory differences (hearing vs. 
deaf) or because the bimodal bilingual's languages exist in two 
different modalities thereby minimizing sensory-motor compe- 
tition (Hyp. 1). Instead, we interpret our results as supporting 
Hypothesis 2. We suggest that the neural demands/impact of early 
bilingual language exposure may influence the fonctional organi- 
zation of the left parietal brain region in ways that, in turn, offer 
a possible explanation of the observed differences between our 
bimodal bilinguals and our native sign language users. 

Neuroimaging studies have found that increased expertise in 
a higher cognitive function lowers activation in brain regions 
dedicated to that fonction (Xue and Poldrack, 2007; Chein and 
Schneider, 2012). Is it possible that lower activation in left pari- 
etal and other regions of the bimodal bilinguals reflects a more 
efficient manner of language processing? Our previous study with 
this exact group of bimodal bilinguals and English monolinguals 
revealed that bimodal bilinguals showed heightened activation 
in their left temporo-parietal regions depending on the bilingual 
language processing context. Bimodal bilinguals showed height- 
ened activation in left temporo-parietal regions during bilingual 
language switching tasks relative to non-switching language con- 
ditions (Kovelman et al, 2009). The present findings show that 
bimodal bilinguals had lower activation in this general region 
during the processing of signs, as compared to Deaf signers. Such 
converging evidence suggests that bilingual exposure impacts lan- 
guage organization in parietal and other left hemisphere regions 
in bimodal bilinguals as compared to functionally-monolingual 
users of sign or speech. 

This study used the case of bimodal bilingualism to address 
the field's core question of whether sensory-motor competition 
between two languages is necessary to change language organi- 
zation in the brain of a bilingual individual (Bialystok, 2001; 
Abutalebi et al., 2008; Emmorey et al., 2008; Kovelman et al, 
2009). The study found both focal (parietal) and broad-spectrum 
differences (especially during pseudo-sign) in left-hemisphere 
activation between bimodal bilinguals and Deaf signers. One 
possible explanation is that the data confirms hypothesis 2: the 
hearing and signing bimodal bilingual experience does impact the 
brain and that the impact of bilingualism extends beyond the 
sensory-motor competition, and reflects an accommodation for 
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learning and using two complex linguistic systems. It is possible 
that the extensive bilingual exercise of left hemisphere language 
regions may have improved their functionality in bilinguals. 
Hence, bimodal bilinguals showed an overall lowering of levels of 
activation throughout the left hemisphere relative to Deaf signers 
and did not show an increase in these regions' activity even as the 
task demands increased. 

Another possible explanation for the findings is that they 
stem not from the hearing, but from the deaf bimodal bilin- 
gual experiences. Even though the Deaf participants in the study 
were functionally monolingual (predominantly relied on ASL for 
their daily communication), research has recently found that Deaf 
signers have greater left hemisphere volume in frontal and motor 
regions relative to hearing signers (Allen et al., 2013). This might 
be due to deaf individuals' accommodation for the complex task 
of mastering spoken language(s) via the visual modality (Allen 
et al., 2013). Thus, on the one hand, the study confirms that bilin- 
gual exposure changes the brain. Yet, on the other hand, the study 
does not provide strong adjudicating evidence of whether it was 
the hearing or the deaf bimodal language learning experience that 
best explains the findings. Future studies will need to simultane- 
ously consider both functional and neuroanatomical evidence to 
better understand the impact of bimodal language experience on 
the language organization in the brain. 

Finally, the present findings found nearly opposing PCA brain 
imaging results for signers and non-signers during the sign repe- 
tition task, even though non-signers were able to achieve nearly 
90% accuracy in repeating the signs. The results suggest that 
brain regions cluster or work together differently for signers vs. 
non-signers during an ASL language task. Prior imaging stud- 
ies with other brain imaging modalities (e.g., fMRI, PET, ERP) 
have consistently demonstrated linguistic vs. non-linguistic pat- 
terns of sign processing by the brain in deaf signers relative to 
hearing non-signers (Hickok et al., 1996; Neville et al., 1997; 
MacSweeney et al., 2002a,b; Corina et al, 2003; Emmorey et al., 
2007), including the study that used the exact same imaging task 
(Petitto et al., 2000). Therefore, we hope to have imparted to 
our readers confidence in the effectiveness of the fNIRS imaging 
method for research on questions involving neural recruitment 
during human language processing. 

One of the study's caveats is the low number of participants 
and hence, the inability to achieve higher statistical thresholds or 
control for multiple comparisons. Native signers are becoming an 
increasingly low incidence population and thus, very challeng- 
ing to recruit and to study — especially involving neuroimaging 
studies in light of the dramatic rise in the availability of auditory 
augmentation technologies (e.g., cochlear implants). The other 
caveat is the relatively large regions of the brain covered by indi- 
vidual fNIRS channel measurements. Nonetheless, the present 
findings provide powerful corroborating evidence with other 
studies of signers, which had used larger samples and technologies 
with better neuroanatomical spatial resolution (e.g., Corina et al., 
2003; Emmorey et al., 2007). Together, all such studies contribute 
to the growing volume of knowledge that both differences in early 
life experience and variation in cross-modal language process- 
ing can have on the neural systems and processing of human 
language. 



CONCLUSION 

Through a novel view provided from individuals who are bilin- 
gual across two different language modalities (signed and spo- 
ken), we have observed the brain's remarkable neural plasticity of 
brain regions underlying human language processing. In partic- 
ular, we gained insight into the multiple and varied conditions 
by which particular brain sites may be recruited. The parietal 
regions that have been viewed as selectively sensitive to sign lan- 
guage phonology and syntax (Newman et al., 2002), as well as 
language switching in bimodal bilingual signers (Kovelman et al., 
2009), also showed a modulation based on the bilingualism status 
of the language user. The present findings also provide support 
for the use of fNIRS technology in studying human brain func- 
tion. Taken together, the findings expand our knowledge of the 
"signing" and the "bilingual brain," and provide new insights into 
the resilience and plasticity of our brain's remarkable capacity for 
human language. 
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